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Abstract 

Using a power sum (boson) realization for the Macdonald operators, we inves- 
tigate the Gukov, Iqbal, Kozgaz and Vafa (GIKV) proposal for the homological 
invariants of the colored Hopf link, which include Khovanov-Rozansky homology 
as a special case. We prove the polynomiality of the invariants obtained by GIKV's 
proposal for arbitrary representations. We derive a closed formula of the invari- 
ants of the colored Hopf link for antisymmetric representations. We argue that a 
little amendment of GIKV's proposal is required to make all the coefficients of the 
polynomial non- negative integers. 



1 Introduction and notation 



In the setup of string theory, the invariants of the colored Hopf hnk are identified as 
topological open string amplitudes on the deformed conifold T*S^ as follows: the U{N) 
Chern-Simons theory is realized by topological string on T*S^ with topological D- 
branes wrapping on the base Lagrangian submanifold S^. The Hopf link in consisting 
of two knots /Ci and /C2 can be introduced by a pair of new D-branes wrapping on 
Lagrangian three-cycles £1 and C2 such that S^nCi = /C, [1]. The topological open string 
amplitude of this brane system is supposed to give the invariants of the Hopf link. The 
coloring or the representation attached to each knot /Cj is related to the boundary states 
of the open string ending on Ci by the Frobenius relation. After geometric transition 
or by the large duality [21 EJ Hj, this brane configuration is mapped to the resolved 
conifold C(-l) © 0{-l) P\ The D-branes wrapping on disappear, but a pair 
of Lagrangian D-branes remains as a remnant of the Hopf link. We can describe the 
resulting D-brane system in terms of the toric diagram and compute the corresponding 
amplitude by the method of a topological vertex [5l E] . 

The aim of this paper is to investigate the conjecture of Gukov, Iqbal, Kozgaz and 
Vafa (GIKV) [7J on the superpolynomial Vx^^{sl, q, t) of the homological invariants of the 
Hopf link colored by two representations A and fi. The superpolynomial of our interest 
is a polynomial in (a, q, t) G C^, such that a specialization a = q^ leads the Poincare 
polynomial of the si{N) link homology 'Ps[(Ar);A,^((q, t), which is a two parameter (q, t) 
version of the sl{N) link invariants. When the coloring is the A^ dimensional defining 
representation, it is called the Khovanov-Rozansky homology In [7], they argued 
the relation of the homological invariants with the refined topological vertex [9l [10] and 
the superpolynomial PA,/i(a, q, t) was expressed as a summation over all the partitions 
(see section 4). However, it is totally unclear whether the proposed expression actually 
gives a polynomial in a and a specialization a = q^ leads the polynomial in q and 
t. Moreover, the coefficients of the polynomial after the specialization are expected 
to be (non- negative) integers due to the following reason. In [TT], it was argued that 
homological link invariants are related to a refinement of the BPS state counting in 
topological open string theory. The GIKV conjecture on homological link invariants of 
the Hopf link was based on this proposal. This means for the Hopf link L that there is 
a doubly graded homology theory 'Hf^^^'^'^{L) whose graded Poincare polynomial is 




(1.1) 
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The physical interpretation of T-L^j '^{L) as the Hilbert space of BPS states leads to 
a prediction on the dependence of the link homologies on the rank — 1. It has been 
conjectured [121 [13] that there exists a superpolynomial V\^fj,{sL,q,t) which is a rational 
function in three variables such that 

VBm;X,M, t) = VxA^ = q"", q, t) . (1.2) 

We see that (11.11) and (II. 2p imply that the specialization a = leads to a polynomial 
with (non-negative) integer coefficients. 
In this paper, we prove the following. 

(i) The GIKV's proposal P^^^f^(a, q, t) for the superpolynomial Pa,/x(cI, q, t) of the 

homological invariants of the colored Hopf link gives really a polynomial in a for 
arbitrary representations. 

(ii) P^^p^(q^, q, t) vanishes for sufficiently small G N, if one representation is anti- 

symmetric and the other is arbitrary. 

(iii) v'^r^J {q^ , q, t) with arbitrary G N is a polynomial in q and t with non-negative 
integer coefficients, if both representations are antisymmetric. 

Furthermore, we perform the summation over the partitions and show a closed formula 
without assuming the condition \t\ < 1 in [H] for the superpolynomial of the homological 
invariants for antisymmetric representations. 

The key ingredients of our proof are the Macdonald operators and a non-standard 
scalar product for the Hall-Littlewood polynomials. The Macdonald operators are a 
set of difference operators which commute with each other on the space of symmetric 
functions. Their simultaneous eigenfunctions are the Macdonald polynomials which are 
two parameter (g, t) deformation of the Schur polynomials |l] By using a power sum 
realization for the Macdonald operators [IHl [ISjjj we investigate the polynomiality and 
integrality of the superpolynomial of the homological invariants obtained by GIKV's 
proposal. 

On the other hand, the Hall-Littlewood polynomial is the symmetric polynomial ob- 
tained from the Macdonald polynomial by letting q = 0. The higher Macdonald operator 
is realized by using a (non-standard) scalar product for which the Hall-Littlewood polyno- 
mials are the basis dual to dominant monomials [17] . We show the pairwise orthogonality 

^We also use another set of parameters {Q,q,t) defined by (a;q,t) — ((5~5;t~^, —{t/q)^). 
^We use a slightly different definition from that in |16) . 
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of the Hall-Littlewood polynomials for this scalar product and use it for the calculation 
of the homological invariants. 

For non-antisymmetric representations, the GIKV's proposal for the superpolynomial 
of the homological invariants calls for some improvement because it has negative integer 
coefficients in general. We find that this positivity problem may be overcome by replacing 
the Schur function by the Macdonald function with an appropriate specialization. 

The paper is organized as follows. In section 121 we discuss two kinds of scalar product 
for the Hall-Littlewood polynomials, which are used in the following sections. In section 
131 we give a power sum realization for the Macdonald operators which are defined in 
appendix C. Our main results are stated and proved in section |H We review the Gukov 
et aFs conjectures on the homological invariants of the colored Hopf link and prove some 
of them. SectionOand appendix F are devoted to the discussion on the positivity problem 
and its example, respectively. Appendix A contains a brief summary of the symmetric 
functions. Proofs of the key theorems are shown in appendices B and D. In appendix E, 
we comment on a relation between the torus knot and the Macdonald polynomial. 

Notations. The following notations are used through this paper. Let A be a Young dia- 
gram, i.e. a partition A = (Ai, A2, ■ ■ ■ ), which is a sequence of non-negative integers such 
that Aj > Aj+i and |A| := < is its conjugate (dual) diagram. £(A) := A^ is 

the length. Let p = (pi, ^2, ■ ' ' ) be the power sum symmetric functions in x = (xi, X2, ■ ■ ■ ) 
defined as Pnix) := X]i>i^r- treat any symmetric function /(x) in x as a function 
in p unless otherwise stated, and sometimes denote it as f{x{p)). P\{x;q,t), Px{x;t), 
s\{x) and ex{x) are the Macdonald, the Hall-Littlewood, the Schur and the elementary 
symmetric function in x, respectively. We define the following specialization: 

:= V(g"^' - l)^^^-^) + — = V q^^H^(h-^^ + (1.3) 

^ t2 — t 2 ^ t2 — t 2 

which is independent of for any A^ > ^(A). We do not have to assume that |t| > 1. 
Let pn{x, y) := p„(x) + Pn{y)] then, 

n_Tn 

PnicqH", Lt-P) = c" V(g"^" - l)r(5-^) + 4 ^, c,LeC. (L4) 

^ t2 — t 2 

For A^ G Z and A^ > £(A), Pn{qH^^P ,t~P) = is the power sum in 

A^ variables {q^^t'^~^^~^}i<i<N- We do not have to use any analytic continuation or 
approximation because we treat any functions as formal power series. For example. 
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U{x,y;q,t) := exp and A(x;t) ■= exp J2n>o ^ ^i<j 

are power series in formal variables Pn{x), Pn{y) and Xi respectively. We also let gx : = 
11(1 j)gA(~l)9'^' and V := (q/t)^. The g-integer [A^]i := and the g-binomial 

coefficient [^]^ '■= 111=1 "^"i-/^' ^.re polynomials in t with non-negative integer coeffi- 
cients for N,r G N. 



2 Scalar product for the Hall-Littlewood polynomial 

First, we discuss two kinds of scalar product for symmetric polynomials, which we will 
use later. In this section, we treat symmetric functions as symmetric polynomials in 
finite number of variables x = (xi, ■ ■ ■ ,xn) by setting Xi = for i > N + 1. For a 
partition A = (Ai, ■ ■ • , Aat), we denote x^ := x^^X2^ ■ ■ ■ x'^ and rrij := #{ A^ | Aj = j }, 
i.e. A = (0"^n™i2™2 . . .) with Ei>o^i = ^- The Hall-Littlewood polynomials Px{x;t) 
are obtained from the Macdonald polynomials P\{x; q, t) defined in appendix A by letting 
q = 0: 

Px{x;t):=Px{x;0,t) 

= vx\i) X] a(x^A(x;t)-i), vxit) ■.= l[[m,]\t 

aeSN j>o 

A(.; := exp {- 5: 5: |U n - E (I) " (2-1) 

L n>0 i<j i J i<j ^ ^ ^ n>0 ^ ^ 

with the symmetric group iSat on variables§| Here x := (xAr,XAr_i, • • ■ [A^]!* := 

\\\t\2\f ■ ■\N\t and [A^]^ := The canonical scalar product (*, *)f is 

defined from (*, in appendix A by 

{Px{x; t), P^{x- t))t := (Pa (a;; 0, t), P,{x- 0, t))o,t = h,^ \{ \{ (2.2) 

We abbreviate it to (Pa, P/^)*- The Cauchy formula ( lA.Sp is now 

5^ — — -Pa(x; t)PA(y; t) = exp <^ ^ Pn{x)pn{y) \ (2.3) 

which is summed over all partitions A. 



^Note that A(a;;t) = Wi^j l^tx^Jx- ^'^^ analytic continuation. 
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For functions / and g in x, let us define a second scalar product as [T7] 



N 



Here x/s are formal parameters, and exp / is defined as the Taylor expansion in /. For a 

fix) denotes the constant term in /, i.e. (L V j fnX"" 

2Trix J 2TTix 

fo. Note that the kernel function A(x;t) is not symmetric in Xj. Then, {f{x),g{x)y^.f = 
{g{x), f{x))'l^.^. Hence, the second scalar product is symmetric only for the symmetric 
functions. The Hall-Littlewood polynomials P\{x\t) with £(A) < are pairwise orthog- 
onal for the second scalar product and we have □ 
Theorem 2.1. 



{Px{x- 1), P^{x; t))%.^, = {P,{x- 1), Pa(x; t))'^, = 5y^,MtV [N]\t. (2.5) 

A proof is given in appendix B. Since Pir{x; t) coincides with the elementary symmet- 
ric polynomial (x) , we obtain {Px{x;t), er{x))%.t = 5x,ir [^]^, with [^]^ := HLi ■ 
Note that from (IA.12P it follows that 



VfU j-^, er{t^) = [-\yul^er,er)u (2.6) 



{er,er)t eNitP) 
which we will use later. 



3 Power sum realization for Macdonald operators 

In this section, we give a power sum (boson) realization for the Macdonald operators 
which are defined in appendix C. Let p = {pi,P2, ■ ' ') be the power sum symmetric 
functions in infinite number of variables x = (xi,X2, ■ ■ ■) defined as Pn{x) '■= Yli>iX^- 
We treat any symmetric function /(x) in x as a function in p unless otherwise stated. 



3.1 Macdonald operators by the power sum 

First, we define a commutative family of the difference operators in power sums, whose 
eigenf unctions are the Macdonald functions. Let z = (zi, ■ ■ ■ ,Zr). For r = 0, 1, 2, ■ ■ ■ , let 
H and H"- be H := Er>o^"^"' 1 and 

"'it was shown in [T7] that the Hall-Littlewood polynomials are the basis dual to dominant monomials. 
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Q=l 

n>0 a=l J 

^:(.) :=exp {E^P^"E-<^>n| = exp (^^(g" - , (3-1) 

Ln>0 a=l J ln>0 «=1 ^" j 

with er{t^) = nl=i from (]A.12p and p* := ^^f^^^g|-• Here Za and p„ are formal 
variables and ^ '^^fi^) denotes the constant term in /. Note that H"^ is written by the 
second scalar product fl2.4l) as = er(t^){(p\{z~^),ip^_{z))'^.^.i. We also denote them 
as H{x), H^{x) and ip{z;x) if they act on the power sums Pn{x) in x, but they are 
independent of the number of variables Xi. 

The Cauchy formula (12. 3p for the Hall-Littlewood function leads to 

^_ V- Pxix{py,t-')P,iz;t-') ^ ^^^^ P,iz-';t-')P^ix{p*y,t-') 

Here P\{x{p) \ t) is the Hall-Littlewood function in terms of the power sums p and ^(A) : = 
is the length of A. Thus, if is also realized by the Hall-Littlewood function as 

= critn p^i^<py^^'"y^}^^^^ (3-3) 



A 

(A)<r 



Then, we have [16] ([15] for r = 1) @ 

Theorem 3.1. The Macdonald function Px{x; q,t) is an eig enf unction for : 

HPx{x;q,t) = Px{x;q,t)Ex, 
H'-Px{x;q,t) = Pxix;q,t)er{qH''), 

E, := exp I - 5^ iz^p„(gV) 1 = Y^w^eriqhn, (3.4) 

L n>0 J r>0 

and therefore, commute with each other [ii*", H'^] = on the space of symmetric 
functions. 

The proof is given by comparing H"^ with the Macdonald operators. 



^Our definition is slightly different from [IB]. Ours is not symmetric in Zq's, and also the integration 
contour may be different. Thus, we give our proof in appendix D. 
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3.2 Action on the Cauchy kernel 

Second, we discuss the properties of which we will use in subsection |4l3. For the 
Cauchy kernel n(x,?/; q,t) in (1A.5I1 . we obtain 

TTf 7\ = ^ {Pn{x)Pn{z) + Pn{ty)p-n{z)) ) , (3.5) 

with pn{z) = J2a=i ^2 5 (neZ). Since 

A{z; t-')ip''{z; xMx, y; q, t) = A(tz-^ t-i)<^'^(t^-^ y)n{x, y; q, t), (3.6) 

with z := {zr, Zr-i, ■ ■ ■ ,Zi), the following important duality holds for any variables x = 
{xi,X2, ■ ■ ■) and y = {yi,y2, ■ ■ ■), even though their numbers of components are different: 

H'{x)U{x, y- q, t) = H%y)U{x, y; q, t). (3.7) 

In a way similar to ( 13. 2p and ( 13. 3p . we obtain 

<f'{z-x)U{x,y;q,t) _ ^ P^{x{p);t-')Px{z;t-^) ^ P^{z-\t-^)P,{ty{p)-t-^) 
Ti{x,y-q,t) 4" (^A,PA)t-i V (^/.,^M)t- 

f(A)<r f{M)<i" 



A 

l{\)<r 



When X = t we hav( 
Proposition 3.2. 

/f'-(x)n(a;,y;g,t) 



n(x,y;g,t) 



= girer{y,r''), (3.10) 

Er>o(~^)''e'-(^'?/) — exp {-X;n>o^^n(^'^)^''/^} and gir := {-lYt~'-^ . 
Proof. From ((331) and PA(t''; t) = 5A,re^(t''), it follows that 

Ihs = erin V Pp (t-^; r') (ti/(p); t'^). (3.11) 

But from ([2J]) and ([02l) . we obtain 

r r 

lhs= (-l)^n^'"5Z^-^(*"')^^(^)- (3.12) 

i=l s=0 

Then, Y7s=o^r-s{x)es{y) = er{x,y) proves the proposition. □ 



^ This proposition was proved in [14] by assuming \t\ < 1. However, here we do not have to assume 

it. 
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3.3 Product of Macdonald operators 

Next we consider the product of the if's which we will use in subsection |4l4. For 
a partition A, let I := £{\), := H^^ H^^ ■ ■ ■ H^' , = {z\,z'2,--- , z\J and z : = 
{zi, ■ ■ ■ , z\x\) ■= {z\, ■ ■ ■ , z\_^, zl,--- , z\^, ■ ■ ■ ,z{,--- , z{^); then, 



A{z; t) = exp 



1 J. 71 



(3.13) 



n>0 i<j 

By the OPE relations (as difference operators) 



i=l 



fl — t~^)(a^ — 1) 



. n>0 



n 



(3.14) 



with z = {zi, ■ ■ ■ ,Zr) and w = {wi, ■ ■ ■ , Wg), we obtain 

(p^'{z^; x) ■ ■ ■ (p^^{z^; x)Il{x, y; q, t) 
U{x,y;q,t) 

Yl ~ ] Pn{x)Pn{z) + Pn{ty)P-n{z) + (?" - 1) Ypn{z')p^ 



. n>0 



exp 



1 -r" 



. n>0 



n 



Pn{x)Pn{z) + Pn{ty)p^n{z) + ^ Pn{qz')P-n{z^ 

i<j 

A{z;t^') 



When {x,y) = {tP,ct^P) with c G C, since Pnit^) = (tt — t"t)"i it follows that 



(3.15) 



l[Aiz^;t-') 



(p^i{z^;x) ■ ■ ■(p^^{z^;x)Il{x,ct'f; q,t) 



1=1 



Il{x,ct-P;q,t) 
1-t 



X=tP 



A{z- 1-') exp <^ I 



A 1 . 

--r I — t^c/Za 



„=1 1 - t ^Z^ 



(3.16) 



. n>0 i<j 

which is a polynomial of degree |A| in c. For abbreviation, we write — z) instead of 
J2n>o Therefore, by taking the constant term in z, we have 
Proposition 3.3. 

H^{x)U{x,ct-P;q,t) 

X=tP 



ex{tP)U{x,ct-P;q,t) 





A 




= 1 



--r dZa 1 — c/Zq-t-i- 1 — Za/ ZjS 



1 - qz'Jz^ 



Za/tZf^fJr^^^j^^l-qzUtz^f, 



(3.17) 



is a polynomial of degree |A| in c and a polynomial in q and 1/t with integer coefficients 

and vanishes when c = 1. 

Proof. The rhs of (13 .17^ reduces to 



/n^(-i)i:(-rxn(-^)i:(i- 

a=l " ^ n=0 a<B ^ P' n=0 ^ ^ 



A, A, 



i<j o_=\ ^=1 



(3.18) 



with |z^| := a + X]fc=i -^fc' ^^^^ ^ polynomial in g and 1/t with integer coefficients. 



When c = 1, since (1 — l/^a) ^„ 



1/zq,, the integrand in (13.171) has no constant 



term in the last variable z\\\] thus, (13.171) vanishes. 
Let 

if^(a;)n(x,cr^;g,t) 



□ 



Z^^{\-clt^-'] 



ex{tP)Ti{x,ct-P;q,t) 



(3.19) 



X=tP 



Then, for example. 



= 1, 

= (1 - cq) - c(l - g)/^^ (3.20) 
Z(i'i'i) = (1 - cg){(l - cg2) - c(l - g)(l + 2g)/t - c(l - g)Vt'} + c\\ - qf/t\ 

Direct calculation of the functions for |A| < 7 suggests the following conjecture. 
Conjecture 3.4. If c = t^ , N e Z with < N < Xi, then ([X7?D vanishes. 



4 Homological link invariants from GIKV conjecture 

In this section, we introduce the conjecture on homological link invariants of the colored 
Hopf link by Gukov, Iqbal, Kozgaz and Vafa [7] and prove some of themj^ 

7 Our (Q,QGlKV.g^i.q gq^als (Q^/q^i, Q; ; q,t) of [7]. 

In [2], PA,p(a;q,t) = ^(-a)|A|+|Ht2|A||HG;,,^(_gGiKV. Ga,^(-QGIkv. t) = Zi-'(Q;9,0 , 

where (a;q,t) (Q^2 ; ^ — (i/g)2). Note that a = and t = — 1 are equivalent to Q ^ and 
q = t, respectively. 
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4.1 GIKV conjecture 

First, we recall the GIKV conjecture and present our main theorem. Following [7] let us 
consider 

Zxu(Q]q,t) ■.= y" Sxiq''tP)sMP) TT ^— — . (4.1) 

Here S\{q^tP) is the Schur function in the power sum Pniq^f)- From (lA.lOp and (lA.Sp . 
it is written by the Macdonald functions P\{x] q,t) defined in appendix A as follows: 

ZxAQ; q. t) = $^(-^-'g)l''IP,v {qP-, t, q)P,{t^- g, t)sx{q'^tP)s,{qHP) 

= E /p^p\ nr^; W^; q, t)s,{qnns,{qnn. (4.2) 

Note that Z,^.{Q;q,t) = U{QtP,t-P;q,t). Then, Z'^^{Q;q,t) := ZxM;q,t)/Z.^.{Q;q,t) 
is a power series in Q and a meromorphic function in g and t. But Z™j^*((5; g, t) is expected 
to give the superpolynomial of the homological invariants and Gukov, Iqbal, Kozgaz and 
Vafa claimed the following results. 
Conjecture 4.1 [7J. 

(1) Z™^^{Q; q, t) is a finite polynomial in Q. 

(2) Z'^^^it^] q, t) vanishes for sufficiently small N G Z>o. 

(3) Z™^*(t^; g, t)(— l)l'^l+l'^l (A^ G Z>o) is a finite polynomial in q and t with integer 
coefficients. 

(4) Z™j^*(t^; g, t)(— l)l'^l+l^l for sufficiently large N coincides with the sl{N) homological 
invariants of the Hopf link colored by the representations A and fi up to an overall factor. 

Note that if the conjecture 4.1.(1) is established, one can easily calculate Z'^'^{t^; g, t) 
for any given (A,/i; A^) and check (2) and (3). In the following subsections, we will show 
part of the GIKV conjecture by proving a series of propositions^ 
Theorem 4.2. 

(i) Conjecture (1) holds for arbitrary (A,/i) {proposition 4.10) . 

(a) Conjecture (2) holds for arbitrary (A,/i) with |A| + \^\ < 7 or ^ = V {propositions 
4.10 and 4.8). 

(Hi) Conjecture (3) holds for (A,/i) = (A, P) with |A| < 7 or A = F {propositions 4.8 and 
4.6). 

{iv) Conjecture (4) holds for (A,yu) = (1*^,1) {proposition 4.6) . 



^In ^4j, we have shown (ii)-(iv) by assummg the condition \t\ < 1. 
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4.2 q = t case 

When q = t, we have 
Proposition 4.3. 

^' = fxfMQl'^ q-')s,{QqV, q-'l (4.3) 
Z,,.[Q;q,q) 

which is a polynomial of degree |A| + in Q. Here fx := Y[{i j)(^xi~'^)q'^'''^'^^^'''' ■ 
Proof. First, we have the following nontrivial identity: 

sxiq^s.iq'^n = fxf, 'Mq'')',!^'') = s^iq^s^iq^+n, (4-4) 

which is proved by sx/^{qP) = Sxv/^v(—q^f) and the cyclic symmetry of the topological 
vertex |6]. Since the above identity and sx{q^) = fxSx'^i—q^), it follows that 

ZxAQ; q, q) = f,Yl Q^'^^Mq^ns./Aq^nsxiqnsniq^^n- (4.5) 



The Cauchy formula ( ]A.5|) and the adding formula ( ]A.4l) yield 

Zx,M q, q) = f.s.iQq^^", q-nmq'^', q'"; q, q)sxiq'')- (4.6) 

But ((2.12) and (5.20) of [21]) 

UiQq^+P,q-P;q,q) ^ SxjQq'^q-^ 
U{QqP,q-P-q,q) Sx{q^) ' ^ ' ' 

which completes the proof. □ 
Note that for G Z and N > i{X), Pn{q^^^^^, q~^) = Eli Thus, 
s^{q'^^'^^P, q~P) is the Schur polynomial in variables {q^^'^^~^}i<i<N, which is a poly- 
nomial in q with non-negative integer coefficients and vanishes for £{X) < N < i{fi). 
Therefore, when Q = q^ , we have 
Proposition 4.4. If N e Z>o, 

^"'^'""'"ttl^ = (-1)I^I-^"^I/a/.^. {{q'^^'-h.<.<N) sx ({g^-}i<.<.) ,(4.8) 

which is a polynomial in q with non-negative integer coefficients and vanishes for < 
A^ < max{£(A), ^(u)}. This coincides with the colored Hopf link invariant by [18] uj) to 
the over all factory 



^This was shown in [7] for N oo. 
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Proof. liO < N < i{X), sx{q^^P,q-P) = 0. If ^(A) < < g"") = 0. 

By the symmetry Zx,^,{Q; q, q) = Z^^x{Q;q,q), we have Zx,^iQ;q,q) = for < iV < 
max{£(A),£(/i)}. On the other hand, if > max{£(A), s^{q^+^+P, q'^) is the Schur 

poljTiomial in variables {q'^''^^~^}i<i<N, which is a polynomial in q with non-negative 
coefficients. □ 
When g 7^ t, it is difficult to calculate Zx,^{Q] q,t) explicitly for lack of the cyclic 
symmetry of the refined topological vertex [7]. 



4.3 (A,^) = (r,l") case 

For X = V and fi = V^, we have 
Proposition 4.5. 

"^'^^ = girgiseriQf^, t-nesiQq^''h^, t""), (4-9) 

which is a polynomial of degree r + s in Q. Here gir := (— 
Proof. Since Sir(x) = Pir(x; q,t) = er{x), it follows that 

Zi,,p(g;g,t) = Yl /p'^p\ Pv{t~'-^q,t)Pv{t'\q,t)PAq''t'\q,t)es{q''n- (4.io) 

The symmetry Pxiff] q, t)P^{qHP; g, t) = P^ifP] g, t)PxiqHP; g, t) in ([OB leads to 

(Q; g, t) = g, t) ^ P,(t-^; g, t)P,{q^''hP; q, 1)6^^"). (4.11) 

^ \-' r?5 ^v)g,t 

By (13. 4p . we can replace es{q^t'') by as follows: 

^isp (Q; g, t) = Pir it"- q, t) ip\. PviQq^'^^tp- g, t)H'P,{x- g, t) (4.12) 

The Cauchy formula ( 1A.5I) and ( 13.101) yield 



(g; g, t) = Pi. (t''; g, t)ff^n(x, gg^^'^^t''; g, t) |, 

= gisP^r{tP- g, t)n(t-^ Qq^^'hf- g, t)e,(gg(i'')t^ t'O- (4.13) 



But ((2.12) and (5.20) of [21]) 



n(gg(i')t^r^;g,t) P,r{QtP,t-P;q,t) 
U{QtP,t-P-q,t) Pir{tP;q,t) ' ^ ' ^ 
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which completes the proof. □ 
Note that for e Z and > ^(A), Pn{qH^+P,t'P) = ^^^^ Thus, 
er{qH^^p, t~P) is the elementary symmetric polynomial in variables {q'^H^'^^~^}i<i<N, 
which is a polynomial in q and t with non-negative integer coefficients and vanishes for 
i{X) < N < r. Therefore, when Q = , we have 
Proposition 4.6. If N e Z>o, 

i-^y^'^pt^^ = (-l)''^V^7re.({f-^}i<,<^)e.({g(i^)^t^+^^}i<,<;v), (4.15) 

which is a polynomial in q and t with non-negative integer coefficients and vanishes for 
< N < max{r, s} {theorem 4.2 {Hi), X = V case). 

When s = 1, up to an overall factor, (I4.15P agrees to the recent result by Yonezawa 
[19] who used the method of matrix factorization developed by Khovanov and Rozansky 
[8] (theorem 4.2 {iv)). 

In [20] , based on an assumption on the computation of the refined topological vertex 
Taki proposed that the homological invariants for the colored Hopf link are written as 

P^fiv"'Q; q'\ t-') := Cy,s,{tP)s,{qhP, Q-H-P)N,,^{Q-\/t- g, t)gx 

= Cx,^Sx{tP)s^{QqH', t-'')N^,.{Q-q,t), 
CA,,:=(-l)l^lQ^^-^l^ll-l (4.16) 

with the denominator factor A^A,^t of Nekrasov's partition function: 

NxAQ;q,t):= U (i -gg^-^t^'^^+^) H {i-Qq^''^'~'r'^''') ■ (4.17) 

(«j)GA (i,i)eM 

For (A,/i) = (r,P), 

r^I'^i{v-'Q;q-\t-') = Ci.,pSi.(t'')e,(Qg(i''V, t~P)N,r^,{Q; q,t), (4.18) 
which coincides with f l4.9p up to the over all factor (— l)^'+*+iQ(^'+*)/2y-2'-s_ 

4.4 (A, fi) = (A, 1^) case 

Since the set of the Macdonald functions {-Pa (a;; q,t)}\x\=d is a basis of the homogeneous 
symmetric functions of degree d, we can write the Schur function s\{x) by the Macdonald 
functions as 

s\{x) = ^Ux,^P^{x;q,t), Ux,f, := ^ Ux,^{q, q){u~^),y,f,{q,t), (4.19) 
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Our {q,t) ^ {q-\t-^) of ^ 
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where MA,^(g, t) is defined by ( lA.Sp . Note that Ux^^ is a rational function in q and t. 
Then, we have 
Proposition 4.7. 

ZxAQ; g t) ^ ^ ^,,,^,P,(gt^ r^; g, t)e.(Qg^t^ r^, (4.20) 

which is a polynomial of degree \\\ + s in Q. Here g\ := Y[(i j)ex(~'^)^^'^''^^'^^ ■ 
Proof. First, we have 

Zx,is{Q;q,t) = t/A,^Z^,p(g; g, t), (4.21) 

where 

Z,,,s{Q-q,t) := J2 /p^p^ Pn{t~'-,q,t)P,{tP-q,t)P,{qn''-q,t)s,s{qnP). (4.22) 

Then, f[OT|) leads to 

^M,p(Q;g,t) = 5^— ^^P,(t-'';g,t)P^(t'';g,t)P,(g'^t'';g,t)sp(g''tO. (4.23) 

We can proceed in the same way as the last subsection. Namely, Sis(a;) = es{x), (13.41) 
and ^JU\f yield 

(Q; ^) = ^m(^'; ^) E /p^p\ ^)^'n(a;; ^) U=t-- 

= P^(t^; q, t)H'Uix, Qq^"- q, t) U=,-p 

= P^if- q, t)g,sesiQqnP, t'^Mt-", Qq^"; q, t). (4.24) 

But from ((2.12) and (5.20) of [21]) 

U{QqHf,t-f;q,t) ^ P^iQtP,t~P; q,t) 
n{QtP,t-P;q,t) P,{tP-q,t) ' ^ ' ^ 

we conclude that 

^'/.^ = gi^g.P.iQt", t-^; g, t)e,{Qqn'', r^). (4.26) 
Z,,.[Q;q,t) 

□ 

Note that for e Z and > £(A), Pf,{qH^+P,t-P) is the Macdonald polynomial 
in variables {q^'t'^^^^''}i<i<N and vanishes for £(\) < N < (.{fi). Therefore, when 
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Q = , we have 
Proposition 4.8. If N e Z>o, 

Z,,,[t ■,q,t) 

= gises{{f-^i<i<N) J2 Ux,^.gM{<i^'^^'t''^''''}i<i<N-, q, t), (4.27) 

^t<A 

which vanishes for < N < max{£(A), s} {theorem 4.2 (ii), fi = V case). 

Finally, we should make a remark on the fact that the transition function Ux^^{q, t) in 
fl4.19p is a rational function in q and t. It is not obvious that the formulas in the above 
propositions are in fact polynomials in q and t. However, we have checked the following 
conjecture up to (i = 7 by direct calculation. 

Conjecture 4.9. For \\\ = d, J2fj,(x>fi>iy) U\^g^{U^^)^^^ is a polynomial with integer 
coefficients of degree d{d — l)/2 in q and of degree d{d — l)/2 in t^^. 
On the assumption that the above conjecture is true, if G Z>o then 

§^ilpM = ,i.e.(t^+^t-) Yl Ux,,g,{U~XMq^''h^^^,t-n (4.28) 

lMl = kl = |A| 

is a polynomial in q and t with integer coefficients because sx{x) is a function in x with 
non-negative integer coefficients (theorem 4.2 {Hi), |A| < 7 case). 

4.5 General (A, fi) case 

From (lA.ip and (lA.Sp . we have 

^^(a;) = Xl^^-MeMv(x), Vx,m:= Y1 «a,^(«"^)^,M' (4.29) 

where ma,^ := Ux^^{q,q) and a^,^ is defined in (lA.ip . Note that = |A| in the above 
equation. More precisely, we have the Jacobi- Trudy formula sx{x) = det [ex^ -i+j{x)) ^^^^ 
with e_r = for r > 0. Then, we have 
Proposition 4.10. 



Zx,Mq,t) y H-\x)H'^\x)U{x,Qt-P-q,t) 

Z.,.iQ;q,t) ^ n{x,Qt~P;q,t) 



(4.30) 



X = tP 



which is a polynomial of degree |A| + in Q and vanishes when Q = 1 {theorem 4.2 {i) 
)• 
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Proof. dUD and (1X5|) yield 



X=tP 



= V Vx,.V^,. H''" {x)H'^" {x)Ii (x, Qt-P; q, t) . (4.31) 



CT<^ 



Then, the proposition in subsection [3l 3 completes the proof. □ 
Although Zx^niQ] q,t) is a power series in Q, we need its partial sum with degree 

|A| + to calculate Zx,^j,{Q; q,t) /Z,^,{Q; q,t). 

On the assumption that the conjecture 3.4 is true, if Q = , N ^ Z with < < 

max{£(A), then (I4.30p would vanish (theorem 4.2 (ii), |A| + < 7 case). 

5 Towards a resolution of positivity problem 

We now make a comment on the positivity problem of the GIKV's proposal. Let 
(a;q, t) := (Q^^;t^^ , —(t/q)^). When a = with N E N, the superpolynomial of 
the homological invariants of the colored Hopf link reduces to q^t-' dim'H^'j^'*'^''^ 

with certain doubly graded homology Tifj^^'^'^ [7]. Therefore, it should be by defini- 
tion a polynomial in q and t with non-negative integer coefficients. However, in general, 
Z^'^^*(t^;g,t)(-l)l^l+l'^l is not so. For example, 

- 47(^2; q, t) = q\f + t^) + q\t' - t') + q{t' + t^). (5.1) 

A solution to this positivity problem may be given by replacing the Schur function in 
(14. ip by the Macdonald function Px{z] q,i) with i = and appropriately chosen q. Let 

Zx,M9^i) ■= $^(-^^-'g)l''lAv(g'';t,g)P,(t'';g,t)P,(g^t'';g,0)P^(g''t'';g,0), (5.2) 

V 

and Z'^;;{Q;q,t) := Zx^Q; /Z.^.iQ; q,t). Note that Z'^;;{Q; q,t)\_^^ = Z'^^{Q-q,t) 

and Z™^p((5; g, t) = ^^'^^(Q; g, t), because ofPA(a;;0,0) = Sx{x) and Pir(2;; g, t) = Sir(x), 
respectively. Since by flA.ip and (IA.3p . 



PA(a;;g,t) = Ef/A,^P^(x;g,t), f^A,/. := Ux,v{q,t){u ^)^^^{q,t), 

= ^Vx^^,e^y{x), Va,^ := ^ Ux,u{q,i){a~^)^^^, (5.3) 
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all propositions in subsections HI 3 and|ll4 hold with these ?7a,^ and Va,^- 

If we choose the parameter q appropriately, it may overcome the positivity problem. 
For example, when q = q, 



-t"^Z'^\e-q,t) 
More generally, for Q = , 



q\e + + + f") + q{q + + t^). 



(5.4) 



q=q 



tiZ'^l\t^-q^t) = qH 



-3+3 



N' 
3 



+ q{q+l)\qQt 



J t 



N' 
2 



J t 



N 
3 



+ q-'QH[N]t + q'Qt{t'-l) 



But if we choose q = q, then the negative coefficient in the g^-term vanishes as 



'N' 




'N' 




'N' 


_ 2 


+ qQt^ 

t 


_ 2 


t 


3 



.(5.5) 



t-^Z^^:\t'';q,t) _ =q^Q'[N], + Q 



q=q 



N' 
2 



[2; 



3 



+ g(g + i) (^Q 

A^ + 2 
3 





'N' 








'N' 












_ 2 




3 




3 



+ g(g + l)t[2; 



'N + 1 




'N' 




3 




3 



(5.6) 



Note that the g- integer [N]t := ^yzr ^"^^ g-binomial coefficient [^]^ := ni=i 
are polynomials in t with non-negative integer coefficients for A^, r G N. We checked that 
(— l)l^l+*Z™p()f:^; g, t)|g=g (A^ G Z>o) is a polynomial in g and t with non-negative integer 
coefficients for |A| -|- s < 5 (see appendix F). 

For non-antisymmetric representations, the specialization g = g fails to solve the 
positivity problem. For example. 



Z'^^{t';q.t)/{qty 



q=q 



qH" [2]t + q-'r (t^ - 1 ) + g^r (t^ + 3t + 1 ) + q^ (t^ + 2t + 3) + [ 3 



(5.7) 



However, if we choose g = (1 + qt)q/t + p with arbitrary p, then 



^2,2 {t -^(l.t)/{qt) 



= pH^ + 2p{qH^ + qtif + t^ + l)+ t^} 

q={l+qt)q/t+p 

+ qh^t'' + 1^ + 1) + qh{t^ + 2e + ^ + 2) + q^t' + + 3t^ + + 1] 



+ qt^{t^ + 2t^ + t + 2) + t'[3[ 



(5. 



We checked that (-l)l^l+I^IZ^'^j^*(t^; g, t)|g=(i+gt)q/t+p (A^ G Z>o) is a polynomial in g, t and 
p with non-negative integer coefficients for |A| -|- |/i| < 5 (see appendix F). Although an in- 
terpretation from the viewpoint of homo logical algebra is unclear, Z™^*((5; g, t) |g=(i+qi)g/t+p 
gives a generalization of the colored Hopf link invariants with a third parameter p in the 
sense that it reduces to Z™^^{Q; g, t) of GIKV when g = 0, i.e. p = —(1 + qt)q/t. 
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Appendix A. Symmetric functions 



Here we recapitulate basic properties of the symmetric functions in x = (xi, X2, ■ ■ • ) [22] . 
The monomial symmetric function is defined by m\ := Y2a ^i"'^^^ ^2'^'^'^'' ' ' ^ where the 
summation is over all distinct permutations of (Ai, A2, ■ ■ ■ )■ The power sum symmetric 
function is Pn '■= X^i^i ^7- '^^^ elementary symmetric function defined by 



■r>0 i>l I n>0 

enjoys 



n 



eAV = ^aA^m^, oaa = 1, ax^, e Z>o. (A.l) 

fi<X 

For any symmetric functions / and g, in power sums Pn's, we define a scalar product as 

1 - g" 9 

(/(P), 9{p))q,t ■■= f{P*)g{p) I constant partK '■= ^ I _ fn Qp " 

The Macdonald symmetric function Px{x;q,t) is uniquely specified by the following 
orthogonality and normalization: 

{Px{x;q,t),P^{x;q,t))g^t = if A /i, (A.2) 
Pxix; g, t) = ^ ux^^iq, t)m^{x), Uxxiq, t) = 1, Ux^iq, t) G Q(g, t). (A.3) 

Here we used the dominance partial ordering on the Young diagrams defined as X > ^ 
|A| = and Ai + ■ ■ ■ + Aj > /xi + ■ ■ ■ + ^Uj for alH. Note that P{x; q~^, t^^) = P{x; q, t). 
The scalar product is given by 

{Pxix; q,t), Pxix; q,t))g^t= [[ ..x^^-i+v 
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We abbreviate it to {Px,Px)q,t- The skew-Macdonald symmetric function P\j^{x]q,t) is 
defined by 

Px/^{x; q, t) := P;(x; q, t) Px{x; q, t)/{P^, P^),^t. 
Let a; = (xi, X2, ■ ■ ■ ) and y = {yi,y2, ■ ■ ■) be two sets of variables. Tlien, we liave 

P\/,,{x; q, t)Pf,/^{y; q, t) = Px/u{x, y; q, t), (A.4) 

where Px/u{x,y; q,t) denotes the skew-Macdonald function in the set of variables 
(xi, X2, ■ ■ ■ ,yi,y2, ■ ■ ■)■ The following Cauchy formula is especially important: 



A 

with 

We denote 



E !p^'p"!''* -Pa/m(^; q, t)Pxiy; g, t) = U{x, y- q, t)P,{y- q, t) (A.5) 



n{x, y; q, t) := exp < ^ _ ^ ^ Pn{x)pn{y) > 

[n>0 ^ q J 



Pn(g V) := y (g"^' - + — = y + (A.6) 

J=l 4=1 

for any N > i{X). Let Pn{x,y) := +Pn(z/); then, 

, „n_rn 

p„(cgV, Lt-'') = c" - l)^^^-) + c, L G C. (A.7) 

^ t2 —t 2 



Note that [10] 



P^v(-t^^g^;t,g) = P,,(g- V^; g, t). (A.8) 



The Macdonald function in the power sums p„ = (1 — L'^)/{t2 — t 2 ) is [22] (Ch. VI. 6) 



for a generic L G C. Note that 



PA(t''; g, t)PAv (g''; t, g) = Px{t-^'- g, t)PAv (g-^; t, g) 



n aVT^T^^ A^- (A-10) 
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If L = t-^ with G N and > £(A), then 



N 



i=l 



is the power sum symmetric polynomial in variables {g^'ta *}i<j<7v; hence, P\{t^, t ^ 
reduces to the Macdonald symmetric polynomial in N variables. Therefore, 

PxitP, r^-P- q,t) = for i{X) > N eN. 

For N eN, there is a symmetry ^2j(Ch. VI. 6) 

Px{tf,t-''-P-q,t)P^{qH'',t-^~P-q,t) = P^{tP,t-''~P;q,t)Px{qHP,t-''-P;q,t). (A.ll) 

The Hall-Littlewood and Schur functions are defined by Px{x]t) := Px{x]0,t) and 
sx{x) := Px{x;q,q), respectively. Note that Pn{x), sx{x) and er{x) = Pir(x;g, t) are 
symmetric functions in x with non-negative integer coefficients. Note that 



^er-six)es{y) = er{x,y). 



s=0 



For A = l^ (ICT reduces to 



i=l 

Note also that 

Px{t';t) = 6x,irer{tn. 
The g-integer and the g-binomial coefficient 



1-t 



N 
r 



^ 1 j-N-r+i 



- t' 



(A.12) 



* i=l 

are polynomials in t with non-negative integer coefficients for A^, r G N. Note that 



AT 
r 
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Appendix B. Proof of ( 12.51 ) 



Here we prove (12.51) . For a G iSat, let dio^) := # { (i, j) | ^ < j, 0(1) > cr(j) } be the 
inversion number and let 



n>0 



n 



E 



X 



T(i)><T(j) 



(B.i; 



Then, 



1 - r 



n>0 



X 



aii) 



1 z<j ^cr{i) 



'<3 ^crU) 



(B.2) 



l~*^g-(j)/^<7(i) 



which is a formal power series in {xjlxi}i^j and is equivalent to crA(x; t) ^ = ni<j Tz^; — /x 
by the analytic continuation. Since P\{x\t) in (12.11) is a polynomial in x, it follows that 



N 



P^{x;t) = v^\t)A{x;t)-' A^JJ^'c 



(0 



(B.3) 



and 



Here i/ is a sequence of integers u = (z^i, z/2, ■ ■ ■ , i^n), z^j € Z and > is the dominance 
partial ordering defined as u > \ J2i = Xli -^j z/i + ■ ■ ■ + z/^ > Ai + • • ■ + for 
all k. Thus, we have 

Vxit){Px{x;t),x^y;,., = l, 
Vx{t){P,{x;t),l[x'^^^^)l, = t'^'^\ 



t;A(t)(PA(a:;t),n<«)'^* = 0' < ^- 

i 

Therefore, by (1A.3P we conclude that 



(B.5) 



v,{t){P,{x;t),P,{x;t))%.^, = 6xAN]h. 



(B.6) 



Here we use the identity X^o-eSjv '^'^^'^^ ~ [^I't; which is proved by the induction in N. 
This completes the proof of (12.51) . 
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Appendix C. Macdonald operators 

Here we define (higher order) Macdonald operators which are compatible with tending 
the number of variables to infinity [IH] [23] [H] . For each integer r such that < r < A^, 
let Z?]y be the Macdonald operators in N variables ), D% := 1 and 

D],:=r(^-)/^5:n^r^n^-.' (c.i) 

I iei ^ J i^i 

summed over all r-element subsets / of {1, 2, ■ ■ ■ , A^}. We set D'j^ := 0, r > N. Here .j, 
is the g-shift operator such that Tq^fi.^) = f{qx). Let Dn{w) := J2^=o^n'^^'^ then, the 
Macdonald polynomial is the eigenfunction for Dn: 

Dn{w)Px{x; q, t) = Px{x; q, t)eN,x, 

N N 

eN,x ■■= Hil + wqH''-') = 5^u;'^e,(g^t^-5+P,tl-P). (C.2) 

i=l r=0 

Therefore, Z)]v simultaneously diagonalized by the Macdonald polynomials 

D'^Pxix; q, t) = P,(x; q, t)e,(g^t^-5+'', t^^; (C.3) 

thus, D]y commute with each other [D^^, D^^^] = on the space of the symmetric function 
in N variables. Note that D'^j^ is not compatible with the restriction of the variables 
defined by setting xn = 0: 

DN{w%^=o = {l + w)DM-i{tw). (C.4) 

So we need to modify it to take N — )■ oo. By using Pnit^~^) = ~1) 
exp {E„>o -n^} = Em>ot-^^"'eUt''), let 



^, w := wt 2, 



ln>0 J r>0 

min(r,-/V) 

Hn= Yl t^'-''''er-s{tnD%, r = 0,l,2,---. (C.5) 



s=0 

Then, by ( 1A.6I) . we obtain 



E^^x := exp i ^^-^ [ e^,, = ^^^e.(gV), (C.6) 

n>0 J r>0 
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which is independent of for any > ^(A). Thus, 



HnPx{x] q, t) = Px{x; q, t)EN^x, 

H'^Pxix; q, t) = Px{x; q, t)e,(gV). (C.7) 



Appendix D. Proof of ( 13.41) 



Tjr 

-Hat 



Here we prove (13 ■4p by comparing with in (IC.Sp . In this subsection, we suppose 
that the number of variables x = (xi, ■ ■ ■ ,xn) is finite by setting Xi = 0, i > N + 1 and 
Pn{x) = J2iLi n E N. For each integer r, we denote H^f := f^^ /erif ), 

a=l a=lj=l ^ ^1^^ a<l3 ^^1 U>0 a=l 

Here Za and pn are formal parameters. For abbreviation, we write 1/(1 — instead of 
Sn>o ^'^ formal parameter z. Note that the operators H^^ are compatible with 

the restriction of the variables defined by setting a; at = 0, -ff]v-i = ^ '^-f^]vUiv=o- We also 
denote := t"''^];rU,=o- 

Instead of the rational function {tXi — Xj)/{xi — Xj), we use 

1 / J 



^ n>0 



^ _^ t x,/x, ^ _ V(x./x,)", z > J. (D.2) 

Then, we have the following recurrence relation. 
Lemma D.l. Suppose Xi ^ Xj if i ^ j . For any N G Z>o and r G 

N 

Proof. For p„ = ^^^x^, n G N, since Tq^iPn = ((?" — l)a^r + Pn), we have for any 
function f in p 

T,,.J{p{x)) = exp /^^(g'^ - 1)^"'^^ 1 /(p(a^))L=.-. (D.4) 

The constant term in Zr is represented as the contour integral surrounding oo and is 
written by the summation of the residues at Zj. = oo and 1/xi with Xj 7^ as follows 
[Fig. 1]: 
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X 

oo 



X 

1 

XN 



X 

XI 



X 



X 

Zr-1 
t 



X 
t 



X 





Figure 1: The constant term in Zr can be represented as the contour integraL 

5«=/n^nn^n^exph:(«"-i)Ec-^ 

^ a=l <^a=lj=l " a<^ Kn>0 a=l 

N „ r-1 , r- 1 ^ 

+(*-i)E/n^n[-inT^v. 



i=l " a=l " a=l 

r-1 N r-1 I ( r-1 „ 

a=lj=l ^J^"a<^ ^"/^^^ U>0 a=l 

=5'-'+«'-'(«-i)E/n^ni'.^i-r„.. 

j=l a=l " jy.t) 

n n mS-p (ec^ - d e ^r^l . (d.) 



X 



Let us denote © {i} e Jj := {1, 2, • • • , A^}. Let 



tt//i.=r- jej^. 



then, we have the following recurrence relation. 
Lemma D.2. For any r = 0, 1, • • • ,N — 1, 



Proof. 



N 

t-^(^-'^/' X ihs = ^ n [ i ] • ^^.-^ E n [ ^ ] n 



h ken keh 
Ui=r leJi 



□ 



D'N-Uk)---t^^''"^'" E \{['n\{T,,xr. (D.6) 



^ j.r+1 _ 1 
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N 

i=i u jehuJi keii k£ii 

N 



i=i h kei kel 

= E EHt^'^int^'^]]!^^.^- (D-8) 

Thus, it is sufficient to show that J2l=i Y[j{j^i)['''^ j] — X]I=o^*' which is proved as follows. 

(i) Since the residues at Xi = Xj vanish, so the Ihs is a constant. 

(ii) By putting Xi = e* and taking e — oo, we obtain the rhs. □ 
Hence, we have 

Lemma D.3. 

r fc— 1 

H^r, = Y.D'^l[if-^-l). (D.9) 

k=0 i=0 

Proof. We proceed by induction on r. When r = 0, since Hp^ = 1, ( ID.Op holds. So 
assume that the result is true for r — 1 > 0. From ( ID. 31) we obtain 

r-l k-1 / N 

k=0 s=0 y i=l 

r-l fe-1 



U^t ^ ' - 1) + if - f-^-^)D%^^) 

=0 s=0 

r— 1 r \ fc— 1 

Y,{f-' - 1) + E(^'^ - ^'■'') D% \[{f ^ - 1). (D.IO) 



,fc=0 fc=l / s=l 



□ 



Therefore, 
Proposition D.4. 

min(r,A'') 

E i^^-'V-.ltO^^, r = 0,l,2,---. (D.ll) 

This completes the proof of (13. 4p by taking the limit — ?■ oo. 

Since the Macdonald functions for all partitions form a basis of the ring of symmetric 
functions, H"^ commute with each other on the space of symmetric functions. 
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Appendix E. Torus knot 



Not only for the Hopf link but the homological invariants for other link may be related 
with the refined topological vertex or the Macdonald polynomial. For example, in [T2] . 
a reduced polynomial for the torus knot Tm,n is conjectured as 

1 - ahu 1 - aHu^ 1 - g^tM"-^ 

with u := (qt)^. But this is nothing but the following specialization of the Macdonald 
polynomial: 



n-2 



i=0 

with (Q; g, t) = {aH/q^; qH^). Note that Pn{Qt^^^, t~^~^) = (1 - - t")- 



Appendix F. Example for Z'^1^{t^] q,t) 

Here we list the example for Z^^^^{t^;q,t) in section [5] with Q = t'^ , which is written 
as a linear combination of the g-binomial coefficients [^]^ with r < |A| + When 
g = (1 + qt)q/t + p for |A| + < 5, we checked that any coefficient of them is a 
polynomial in g, t and p with non-negative integer coefficients, so is Z^^^^{t'^;q,t). The 
same is true when q = q, i.e. p = g — (1 + qt)q/t, for (A, /i) = (A, 1"*) with |A| + s < 5. 



- t2 



'vinst 
^2,12 

7inst 
■^^2 12 



inst 



'71ns 
^2,1 

'vinst 
^12, 1> 





'7inst ) 
^12,,/ 



g g + g(t- 1) + [2]t g + t 
1 1 



q + qt[2]t + t {q + q{t-l))[2]t + [3]t q + t 
1 1 1 



qQt[N]t 



(F.l) 
(F.2) 

(F.3) 
(F.4) 



(F.5) 
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-r2 



y'mst 
f 2,13 
yinst 

^12,13^ 



{qt + l)t {q + q{t-l))[3]t + [4]t q + t 
1 [3]t 1 



mil 



qQ 

t2 



(F.6) 

Note that [4]* = {t^ + l)[2]t. Since {q + q{t - l)}[2]t with g = g + p or (1 + qt)q/t+p 
is a polynomial in g, t and p with positive integer coefficients, so is Z^^ll for |A| = 2 and 
s = 0,1,2,3. 



3 

-t2 



yinst 

'Ziiist , 
\ ^13,. / 



3,0 



t2 



/ '7inst \ 
/ J^3,l2 \ 

yinst 

^^2,1), 12 

V J 



3,2 



/?'<3'[21.[?],\ 

f |3].[4],[^] 



with Ma^s and M4^o in the next page. Since 



/ 3"* \ 

yinst 

, '7inst , 
\ ^13,1 / 



yinst 

'/inst 
_ 22,. 
yinst 

(2,12),^ 

'7inst 
\ ^14,. / 



(F.7) 
(F.8) 



{g + g(t2 - l)}[3]i, g(g + l)(gt + 1) + q\t - \){qt + [2]^, 
{f + (%~+ 1) + - - 1) + [2]t)[2]t}[3]t, 



(F.9) 



with g = g + p or (l + gf)g/i + p are polynomials in g, i and p with positive integer 
coefficients, so is Z^"^ for |A| = 3 and s = 0, 1,2. The same is true for |A| =4,5 and 
|A| + s < 5 with g = g or (1 + qt)q/t-\-p. 



i^Zg^lt^ g, t) = g^^gM ]* + + ^)'^'(^' + 1) [ 3 ] 
+ gV{{2gi + g(gi+[2]i)(i-l)}g[2]i + (g~2 + if + (gi[2]i + l)[2]J 
+ {(g + + g(^-l) + [2 W[2]i + g2(i2 + l) + i([3], + 1)} [3]t 



. t 

N' 



(F.IO) 



Since {g + g(t-l)}[2]t, {g + g(t-l)}[3]t, gt + (t- l)g(gt + [2]^, with g = (l + gt)g/t+p 
are polynomials in g, t and p with positive integer coefficients, so is The same is 

true for (A,/i) = (3,2) and ((2,1), 2) with (1 + gt)g/t + p. 
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3,0 



/ qt \ 

q{q+l) + q{t-l)+t 1 

\ q' + q{q + l)t[2]t + t'' q + t[2]t 1 / 

/ r/2 \ 

qiq + l)iqt+l) + q\t-l){qt+[2]t) + {q[2]t + l)t qt + 1 

q' + m+l) + q{t-mqit-l) + [2]t)[2]t + t[3]t q + {q{t - 1) + [2]t)[2]t 1 

V q' + q{q+l)t[2]t + t' q + t[2]t 1 / 



00 



3,2 



q{q + 1) + g2t2 + _ i) + ^ 



g3 + g(g+l)(g(t2+t_i) + [2]t)[2]j + gt(t-l)(g[2]t(gt+[2]i) + l) + gt3(t+2)+t[3]i g+g(t2+t_i)[2],+ [2; 



g3 + {q{q + 1) + g(t - l))(g(t2 _ 1) + [3]^) + t{t^ + 1) 
g3 + g(g + l)t[2]i + t3 



q + q{t^-l) + [3]t 
q + t[2]t 



\ 
1 
1 

1 / 



*M4,0 = 

/ qH 0\ 

g4 + g2 + ^[3]^^(<^[2]* + l)t + g2(t_l)[2]t(gt+[2]t) + (g[2]t + l)i2 q+[2]ttq + t 1 

g2f(g[3],- + (g2 + l)f) + ((t-l)g + t)(g[3],- + (t-l)g)[2]t+t3 q2 + [2]t{q+it-l)q + t) q + t 1 

V q^ + q[3h{q''+t'')t[3]t + iq^ + q''){t^ + f)+t'' q^ + q{q + l)t[3]t + qt''{t'' + 1) + t^[3]t q"" + qt[3]t + + ^ q + t[3]t ij 



t-^Z^l),2{t -^Q^t) = {q + t[2]t){q + t){t' + l)t'[5]t 
q'Q' 



N 
5 



+ ^-^{q + q{t-l) + qH' + n}[2], 



N' 
2 



+ 



{f + q{q + qH^ + {qt+l)[2]t}{t-l) + q{qt\t+2) + [3]t + 2t} + {[3]t + qt^[2]t)[2]t}[3]t 



N 
3 



+ gQ{{g' + ?(?[2Mt-l) + ?(t+2) + [2],^)(i-l)}[3], + ?~([3]/ + [4],)+t([3],^ + [2],)}i(i2 + l)[2], 



J t 
N' 



t-^Z^^{t''-q,t)^q''Q' 
q'Q' 



N' 
1 



t+{q^ + qH{t + 2) + qH[2]t' + qf{2t + 1) + t^} {t^ + l)t^[5]t 



J t 



N' 
5 



+ 



+ 



t 



q'Q' 



{f + q{q{q + 2) + {{qH + l)(qt + [2l) + qH[2]t)} (t-1) 

+f{qH^ + t + 2) + q{2qH^ + 2t + 1) + {qH^[2 ]t + [ 3 ]*) } [ 2 ; 

{q' + q {qt(qt + [ 2 ],) + [ 2 ],} (f + q\t-l))(t-l)[2]t 



N' 
2 



+ g{g'(g + l)[2]t + g((2gt(?t + [2]i) + l)[2]t + 2t) + ((2gV + g[2]t' + 2M2]* + l)}(i-l) 

+ {g3(gt2(^+2) + [3]i+2t) + g2([3]i + [2]i + gt2(t+2))[2]i + g((2t+l)[3]i + gt3(2t+5)) + (gt2[2],2 + [4]^+t)t}}[3]^ 

+ qQ{q'[2U3Ut-lf + q\q{q+2) + [2]t)[2U3]t{t-l)^ + q{^^ 

+?~^[3], + g=^([3],^ + [4],) + g([4M2], + [3Mi^ + 2))i + ([4], + i)[2],Y + ([5]t + t[2],)t^}i[4] 



N 
3 
it 
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